Refining Sparse Principal Components

Principal component analysis (PCA) is a well-established tool for making sense of high dimensional data by reducing it to a smaller dimension. Its extension to sparse principal component analysisprincipal component analysis!sparce, which provides a sparse low-dimensional representation of the data, has attracted alot of interest in recent years (see, e.g., [1, 2, 3, 5, 6, 7, 8, 9]). In many applications, it is in fact worth to sacrifice some of the explained variance to obtain components composed only from a small number of the original variables, and which are therefore more easily interpretable. Although PCA is, from a computational point of view, equivalent to a singular value decomposition, sparse PCA is a much more difficult problem of NP-hard complexity [8]. Given a data matrix A ∈ Rm×n encoding m samples of n variables, most algorithms for sparse PCA compute a unit-norm loading vector z ∈ R that is only locally optimal for an optimization problem aiming at maximizing explained variance penalized for the number of non-zero loadings. This is in particular the case of the SCoTLASS [7], the SPCA [10], the rSVD [9] and the GPower [5] algorithms. Convex relaxationsconvex relaxation have been proposed in parallel for some of these formulations [2, 1]. To this end, the unit-norm loading vector z ∈ R is lifted into a symmetric, positive semidefinite, rank-one matrix Z = zz with unit trace. The relaxation consists of removing the rankone constraintrank-one!constraint and accepting any matrix of the spectahedronspectahedron